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(54) Consistency checking for a database management system 



(57) The present invention is directed a logical con- 
sistency checker (LCC) working alone or in conjunction 
with a physical consistency checker (PCC) and/or a data 
reliability system (DRS) for a database files system of a 
hardware/software interface system. Logical data cor- 
rection pertains to logical data corruptions for entities 
(e.g., items, extensions, and/or relationships in an item- 
based operating system, where an item-based operat- 
ing system is one example of an item-based hardware/ 
software interface system). In this regard, a LCC anal- 
yses and corrects logical damage to entities represent- 
atively stored in the data store in order to ensure that all 
such entities in said data store are both consistent and 
conform to the data model rules. 
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Description 
CROSS-REFERENCE 

[0001] This application is a continuation-in-part of U. 
S. Patent Application No. 10/837,932 (Atty. Docket No. 
MSFT-3842), filed on May 3, 2004, entitled "SYSTEMS 
AND METHODS FOR AUTOMATIC DATABASE OR 
FILE SYSTEM MAINTENANCE AND REPAIR". 

TECHNICAL FIELD 

[0002] The present invention relates generally to file 
system management and, more particularly, to automat- 
ic file system maintenance and repair to ensure data re- 
liability and consistency with regard to a data model. 
Various aspects of the present invention pertain to re- 
sponding to and correcting logical data errors at a data 
entity level without losing other, down-level (child) data 
entities. In particular, various aspects of the present in- 
vention pertain specifically to the maintenance of logical 
data in an item-based hardware/software interface sys- 
tem. 

BACKGROUND 

[0003] While client database platforms (i.e., home 
and business desktop computers) use hardware of a 
qualitythatismuch lowerthan on server platforms, even 
server-class hardware (controllers, drivers, disks, and 
so forth) can cause "physical" data corruption such that 
a read operation does not return what the database ap- 
plication wrote to the data store. Of course, this is clearly 
a more prolific problem with client database platforms 
(as opposed to server database platforms) for various 
reasons including without limitation the increased prob- 
ability of a client machine been arbitrarily powered off in 
the midst of a write operation due to an unexpected pow- 
er outage (which in turn leads to torn pages and potential 
database corruptions) whereas it is more common for 
server database systems to utilize uninterruptible power 
supplies to mitigate problems from power outages. Me- 
dia decay is another source of "physical" data corrup- 
tions, where the physical storage media quite literally 
wears out over time. And yet another source of concern 
regarding reliability is the detection and recovery from 
"logical" corruptions caused by software errors whether 
inadvertent (e.g., bugs) or pernicious (e.g., viruses). 
[0004] Traditionally maintenance and repair of a da- 
tabases (and database file systems) has fallen to data- 
base managers and the like having a well-developed 
skill set and deep knowledge of database systems, or 
at least to individuals who are familiar with and regularly 
use database systems — by and large persons relatively 
skilled with regard to database technologies. On the oth- 
er hand, typical consumer and business end-users of 
operating systems and application programs rarely work 
with databases and are largely ill-equipped to deal with 



database maintenance and repair issues. 
[0005] While the disparate level of skill between these 
two groups has been largely irrelevant in the past, a da- 
tabase-implemented file system for a hardware/soft- 

5 ware interface system creates a scenario where these 
lesser-skilled end-users will be faced with database 
maintenance and repair issues they will largely be una- 
ble to resolve. Thus a business/consumer database-im- 
plemented operating system file system, or "database 

10 file system" (DBFS) for short, must be able to detect cor- 
ruptions and recover its databases to a transactional^ 
consistent state and, in the cases of unrecoverable data 
loss, the DBFS must then guarantee logical data con- 
sistency at the level atomic change units to said data 

15 are maintained (i.e., at the "item" level for an item-based 
DBFS). Moreover, for DBFSs running by default in a lazy 
commit mode, the durability of transactions committed 
just before an abnormal shutdown is not guaranteed and 
must be accounted for and corrected. 

20 [0006] Moreover, while business/consumer end-us- 
ers will greatly benefit from automating DBFS mainte- 
nance and recovery, database managers and those of 
greater database skills will also benefit from a technical 
solution for general database maintenance and repair. 

25 it is commonplace in the art for database administrators 
to utilize database tools (for example, the database tun- 
ing advisor provided with SQL Server 2000), but these 
tools do not directly address reliability but instead pro- 
vide a means by which backups of the database are ad- 

30 ministered and managed — and not in a mostly-auto- 
mated fashion, but instead requiring substantial data- 
base administrator involvement, particularly when data- 
base backups are not available or other repair issues 
arise. Thus an automated solution to address database 

35 reliability would also be beneficial for database admin- 
istrators and other skilled database users. 
[0007] A data reliability system (DRS) for a DBFS 
comprises a framework and a set of policies for perform- 
ing database administration (DBA) tasks automatically 

40 and with little or no direct involvement by an end-user 
(and thus is essentially transparent to said end-user). 
For several embodiments, the DRS framework imple- 
ments mechanisms for plugging error and event notifi- 
cations, policies, and error/event handling algorithms in- 

45 to the DRS. More particularly, for these embodiments 
DRS is a background thread that is in charge of main- 
taining and repairing the DBFS in the background, and 
thus at the highest level the DRS guards and maintains 
the overall health of the DBFS. For certain embodi- 

50 ments, the DRS comprises the following features with 
regard to physical data corruption: (1) responding and 
correcting data corruptions at a page level for all page 
types; and (2) attempting a second level of recovery (re- 
build or restore) for index page corruptions (clustered 

55 and non-clustered), data page corruptions, and page 
corruptions in the log file. Thus, for certain embodi- 
ments, the DRS comprising functionality for: (i) handling 
repair/restore data corruption cases; (ii) improving the 
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reliability and availability of the system; and (iii) keeping 
a DRS error/event history table for a skilled third party 
to troubleshoot database or storage engine problems if 
necessary. 

[0008] While embodiments may address physical da- 
ta corruption (i.e., correcting corrupted data in a data- 
base stored on the physical storage medium), a robust 
DRS should also address logical data corruptions to en- 
tities (e.g., items, extensions, and/or relationships) rep- 
resentatively stored in the data store in order to ensure 
that all such entities in said data store are both consist- 
ent and conform to the data model rules. 

SUMMARY 

[0009] Various embodiments of the present invention 
are directed a data reliability system (DRS) for a DBFS, 
said DBFS comprising a file system (logical data) main- 
tained in a database (physical data) or, stated another 
way, comprising a database (physical data) that repre- 
sents a file system (logical data). The DRS may com- 
prise a framework and a set of policies for performing 
database administration (DBA) tasks automatically and 
with little or no direct involvement by an end-user (and 
thus is essentially transparent to said end-user). The 
DRS framework implements mechanisms for plugging 
error and event notifications, policies, and error/event 
handling algorithms into the DRS. More particularly, for 
these embodiments DRS is a background thread that is 
in charge of maintaining and repairing the DBFS in the 
background, and thus at the highest level the DRS 
guards and maintains the overall health of the DBFS. 
[0010] For various embodiments of the present inven- 
tion, the DRS comprises the following features: 

• Physical Data Correction: responding to and cor- 
recting physical data corruptions at a page level for 
all page types, and which may include attempts to 
rebuild or restore operations for index page corrup- 
tions (clustered and non-clustered), data page cor- 
ruptions, and page corruptions in the log file. 

• Logical Data Correction: responding to and correct- 
ing logical data corruptions for "entities," e.g., items, 
extensions, and/or relationships in an item-based 
operating system (an item-based operating system 
being one example of an item-based hardware/soft- 
ware interface system). 

[0011] In regard to the second bullet, several embod- 
iments of the present invention are specifically directed 
to a logical consistency checker (LCC) that analyses 
and corrects logical "damage" to entities (e.g., items, ex- 
tensions, and/or relationships) representatively stored 
in the data store in order to ensure that all such entities 
in said data store are both consistent and conform to the 
data model rules. For certain embodiments the LCC 
may be autonomous, while for other embodiments it 
may be coupled with a physical consistency checker 



(PCC) for detecting and correcting physical data corrup- 
tions, and/or for yet other embodiments the LCC may 
comprise a component of a DRS. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

[0012] The foregoing summary, as well as the follow- 
ing detailed description of preferred embodiments, is 
better understood when read in conjunction with the ap- 
10 pended drawings. For the purpose of illustrating the in- 
vention, there is shown in the drawings exemplary con- 
structions of the invention; however, the invention is not 
limited to the specific methods and instrumentalities dis- 
closed. In the drawings: 
15 [0013] Fig. 1 is a block diagram representing a com- 
puter system in which aspects of the present invention 
may be incorporated; 

[0014] Fig. 2 is a block diagram illustrating the struc- 
ture of the data reliability system (DRS) in database file 
20 system (DBFS) representative of several embodiments 
of the present invention; 

[0015] Fig. 3 is a process flow diagram illustrating an 
approach by which logically corrupted entities are as- 
certained for certain embodiments of the present inven- 
ts tion; 

[0016] Fig. 4 is a process flow diagram illustrating a 
three-prong approach for an LCC to resolve logical er- 
rors in an entity for certain embodiments of the present 
invention; 

30 [0017] Figs. 5A and 5B are block diagrams that illus- 
trate the replacement methodology regarding item enti- 
ties for certain embodiments of the present invention; 
and 

[0018] Figs. 6A and 6B are block diagrams that illus- 
35 trate the replacement methodology regarding relation- 
ship entities for certain embodiments of the present in- 
vention. 

DETAILED DESCRIPTION 

40 

[001 9] The subject matter is described with specificity 
to meet statutory requirements. However, the descrip- 
tion itself is not intended to limit the scope of this patent. 
Rather, the inventors have contemplated that the 

45 claimed subject matter might also be embodied in other 
ways, to include different steps or combinations of steps 
similar to the ones described in this document, in con- 
junction with other present or future technologies. More- 
over, although the term "step" may be used herein to 

50 connote different elements of methods employed, the 
term should not be interpreted as implying any particular 
order among or between various steps herein disclosed 
unless and except when the order of individual steps is 
explicitly described. 

55 [0020] The above summary provides an overview of 
the features of the invention. A detailed description of 
one embodiment of the invention follows. For various 
embodiments described below, the features of the 
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present invention are described as implemented in the 
MICROSOFT SQL SERVER database system (some- 
times referred to herein simply as "SQL") alone or incor- 
porated into the MICROSOFT WinFS file system for the 
next generation personal computer operating system 
(commonly referred to as "Windows Longhorn" or 
"Longhorn" for short). As mentioned above, SQL SERV- 
ER incorporates the MICROSOFT.NET Common Lan- 
guage Runtime (CLR) to enable managed code to be 
written and executed to operate on the data store of a 
SQL SERVER database. While the embodiment de- 
scribed below operates in this context, it is understood 
that the present invention is by no means limited to im- 
plementation in the SQL SERVER product. Rather, the 
present invention can be implemented in any database 
system that supports the execution of object-oriented 
programming code to operate on a database store, such 
as object oriented database systems and relational da- 
tabase systems with object relational extensions. Ac- 
cordingly, it is understood that the present invention is 
not limited to the particular embodiment described be- 
low, but is intended to cover all modifications that are 
within the spirit and scope of the invention as defined by 
the appended claims. 

Computer Environment 

[0021 ] Numerous embodiments of the present inven- 
tion may execute on a computer. Fig. 1 andthefollowing 
discussion is intended to provide a brief general descrip- 
tion of a suitable computing environment in which the 
invention may be implemented. Although not required, 
the invention will be described in the general context of 
computer executable instructions, such as program 
modules, being executed by a computer, such as a client 
workstation or a server. Generally, program modules in- 
clude routines, programs, objects, components, data 
structures and the like that perform particular tasks or 
implement particular abstract data types. Moreover, 
those skilled in the art will appreciate that the invention 
may be practiced with other computer system configu- 
rations, including hand held devices, multi processor 
systems, microprocessor based or programmable con- 
sumer electronics, network PCs, minicomputers, main- 
frame computers and the like. The invention may also 
be practiced in distributed computing environments 
where tasks are performed by remote processing devic- 
es that are linked through a communications network. 
In a distributed computing environment, program mod- 
ules may be located in both local and remote memory 
storage devices. 

[0022] As shown in Fig. 1 , an exemplary general pur- 
pose computing system includes a conventional per- 
sonal computer 20 or the like, including a processing 
unit 21 , a system memory 22, and a system bus 23 that 
couples various system components including the sys- 
tem memory to the processing unit 21 . The system bus 
23 may be any of several types of bus structures includ- 



ing a memory bus or memory controller, a peripheral 
bus, and a local bus using any of a variety of bus archi- 
tectures. The system memory includes read only mem- 
ory (ROM) 24 and random access memory (RAM) 25. 
5 A basic input/output system 26 (BIOS), containing the 
basic routines that help to transfer information between 
elements within the personal computer 20, such as dur- 
ing start up, is stored in ROM 24. The personal computer 
20 may further include a hard disk drive 27 for reading 
from and writing to a hard disk, not shown, a magnetic 
disk drive 28 for reading from or writing to a removable 
magnetic disk29, and an optical diskdrive30for reading 
from or writing to a removable optical disk 31 such as a 
CD ROM or other optical media. The hard disk drive 27, 
magnetic diskdrive 28, and optical diskdrive30 arecon- 
nected to the system bus 23 by a hard disk drive inter- 
face 32, a magnetic disk drive interface 33, and an op- 
tical drive interface 34, respectively. The drives and their 
associated computer readable media provide non vola- 
tile storage of computer readable instructions, data 
structures, program modules and other data for the per- 
sonal computer 20. Although the exemplary environ- 
ment described herein employs a hard disk, a remova- 
ble magnetic disk 29 and a removable optical disk 31 , it 
should be appreciated by those skilled in the artthat oth- 
er types of computer readable media which can store 
data that is accessible by a computer, such as magnetic 
cassettes, flash memory cards, digital video disks, Ber- 
noulli cartridges, random access memories (RAMs), 
read only memories (ROMs) and the like may also be 
used in the exemplary operating environment. 
[0023] A number of program modules may be stored 
on the hard disk, magnetic disk29, optical disk 31 , ROM 
24 or RAM 25, including an operating system 35, one 
or more application programs 36, other program mod- 
ules 37 and program data 38. A user may enter com- 
mands and information into the personal computer 20 
through input devices such as a keyboard 40 and point- 
ing device 42. Other input devices (not shown) may in- 
clude a microphone, joystick, game pad, satellite disk, 
scanner or the like. These and other input devices are 
often connected to the processing unit 21 through a se- 
rial port interface 46 that is coupled to the system bus, 
but may be connected by other interfaces, such as a 
parallel port, game port or universal serial bus (USB). A 
monitor 47 or other type of display device is also con- 
nected to the system bus 23 via an interface, such as a 
video adapter 48. In addition to the monitor47, personal 
computers typically include other peripheral output de- 
vices (not shown), such as speakers and printers. The 
exemplary system of Fig. 1 also includes a host adapter 
55, Small Computer System Interface (SCSI) bus 56, 
and an external storage device 62 connected to the SC- 
SI bus 56. 

[0024] The personal computer 20 may operate in a 
networked environment using logical connections to 
one or more remote computers, such as a remote com- 
puter 49. The remote computer 49 may be another per- 
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sonal computer, a server, a router, a network PC, a peer 
device or other common network node, and typically in- 
cludes many or all of the elements described above rel- 
ative to the personal computer 20, although only a mem- 
ory storage device 50 has been illustrated in Fig. 1 . The 
logical connections depicted in Fig. 1 include a local ar- 
ea network (LAN) 51 and a wide area network (WAN) 
52. Such networking environments are commonplace in 
offices, enterprise wide computer networks, intranets 
and the Internet. 

[0025] When used in a LAN networking environment, 
the personal computer 20 is connected to the LAN 51 
through a network interface or adapter 53. When used 
in a WAN networking environment, the personal com- 
puter 20 typically includes a modem 54 or other means 
for establishing communications over the wide area net- 
work 52, such as the Internet. The modem 54, which 
may be internal or external, is connected to the system 
bus 23 via the serial port interface 46. In a networked 
environment, program modules depicted relative to the 
personal computer 20, or portions thereof, may be 
stored in the remote memory storage device. It will be 
appreciated that the network connections shown are ex- 
emplary and other means of establishing a communica- 
tions link between the computers may be used. 
[0026] While it is envisioned that numerous embodi- 
ments of the present invention are particularly well-suit- 
ed for computerized systems, nothing in this document 
is intended to limit the invention to such embodiments. 
On the contrary, as used herein the term "computer sys- 
tem" is intended to encompass any and all devices ca- 
pable of storing and processing information and/or ca- 
pable of using the stored information to control the be- 
havior or execution of the device itself, regardless of 
whether such devices are electronic, mechanical, logi- 
cal, or virtual in nature. 

Overview of the Data Reliability System (DRS) 

[0027] For several embodiments of the present inven- 
tion, the data reliability system (DRS) is a thread that 
maintains and repairs the database in the background, 
and thereby guards the general health of the database 
file system (DBFS). Fig. 2 is a block diagram illustrating 
the structure of the DRS in the DBFS. In the figure, an 
operating system 202 providing operating system level 
services to a plurality of applications 232, 21 4, and 21 6, 
comprises a DBFS 222 logically coupled to a persistent 
data store 232. The operating system 202 further com- 
prises a DRS 242 which is invoked 244 by the DBFS 
222 whenever, for example, a page error 240 from 
among a plurality of pages 234, 236, and 238 in the per- 
sistent data store 232 is discovered, and the DRS 242 
then performs repair operations in response to the page 
error 240. 

[0028] Certain embodiments of the present invention 
provide that the DRS be extensible so that recovery pol- 
icies and detection mechanisms may be updated after 



a DBFS has been released. Several embodiments are 
direct to a DRS that run repairs while the DBFS data- 
base is kept online. Still other embodiments are directed 
to run with full access to the DBFS store (that is, sysad- 

5 min privileges). Still other embodiments will have the 
ability to detect and react to failures in real time. 
[0029] For several embodiments, DRS repairs will be 
transactional at the level change units to said data are 
maintained (i.e., at the "item" level for an item-based 

10 DBFS). For various embodiments, repairs will either 
completely recover an item or it will back out its changes 
(and thus never partially correct an error), and the DRS 
may also have the ability to continue the recovery/res- 
toration work even if a reboot occurs half way thru the 

15 process. For several embodiments of the present inven- 
tion, the DRS will subscribe to SQL events so that if SQL 
fires a general event, the DRS may intercept it and react 
(including without limitation 823/824 events). In addi- 
tion, certain embodiments of the present invention are 

20 directed to a database engine that may be modified to 
send DRS-specific events for error conditions that the 
DRS is to specifically handle. 

[0030] For various embodiments of the present inven- 
tion, physical and/or logical corruptions will be detected 

25 whenever the DBFS reads or writes pages from disk, in 
which case SQL will then generate one of a host of er- 
rors depending on what type of corruption it is and will 
also fire specific DRS events to notify it of the specific 
error conditions, and the DRS will receive those errors 

30 and place them on in an incoming queueforprocessing. 
[0031 ] For several embodiments of the present inven- 
tion, ascertaining whether a page is physically corrupted 
may be accomplished by various means including, with- 
out limitation, (a) examining the checksum for a page 

35 and, if the checksum is invalid, the page is considered 
corrupt or (b) by examining the log serial number (LSN) 
to see if it is beyond the end of the log file (where an 
LSN is an integer that is incremented with each trans- 
action so that if the last transaction in the log was LSN 

40 432 and a page with a greater LSN is found then an out 
of order write error must have occurred.. In this regard, 
there are four major types of page corruptions that can 
effect the operation of a DBFS (in addition to other 
sources such as bugs, etc.), and these four types in- 

45 elude torn pages, media decay, hardware failure, and 
out-of-order writes. Torn pages occur when a page of 
data is not correctly written atomically, and thus any part 
of the page may be corrupted because during a write 
only some of the sectors of a page make it to disk before 

50 the failure event, for example, a powerfailure or a sector 
write failure. Media decay occurs when a data pages 
bits have been corrupted by physical media decay. A 
hardware failure could arise for a variety of reasons re- 
lated to the bus, the controller, or the hard disk device. 

55 As for out-of-order write, these errors stem from the fact 
that IDE drives cannot guarantee the order of writes to 
the disk, especially the IDE drive has write-caching en- 
abled (turned on), and thus it is possible that writes to 
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the data store may occur out of order. If a partial series 
of out of order writes occur but are interrupted by a pow- 
er failure, for example, then several errors may occur, 
such as the data page being written to disk before the 
associated log entry being written for example. While 
out-of-order errors can be detected by checking the log 
sequence numbers (LSN) on data pages, there is no 
easy way to do this short of reading every page. 

Logical Consistency Checker 

[0032] Various embodiments of the present invention 
are specifically directed to a logical consistency checker 
(LCC) that analyses and corrects logical "damage" to 
entities (e.g., items, extensions, and/or relationships) 
representatively stored in the data store in order to en- 
sure that all such entities in said data store are both con- 
sistent and conform to the data model rules. For certain 
embodiments the LCC may be autonomous, while for 
other embodiments it may be coupled with a physical 
consistency checker (PCC) for detecting and correcting 
physical data corruptions, and/or for yet other embodi- 
ments the LCC may comprise a component of a DRS. 
[0033] For a file system built with database technolo- 
gy (a database file system), logical consistency is dis- 
tinct and separate from physical consistency in the 
sense that the latter (physical consistency) refers to the 
database structure itself and the physical storage of that 
database on a storage medium whereas the former (log- 
ical consistency) refers to the logical data schema that 
is represented by the data stored in said database and 
represents the file system of the hardware/software in- 
terface system. 

[0034] Although physical consistency is related to log- 
ical consistency in certain regards (as discussed herein 
below), certain embodiments of the present invention 
are primarily directed to ensuring logical consistency. Of 
course, physical damage resulting in physical inconsist- 
ency (e.g. , a disk sector goes bad, said disk sector con- 
taining a portion of said database structure) may also 
result in damageto logical consistency (e.g., loss of data 
for an entity stored in said database at said bad disk 
sector), but not all logical damage necessarily corre- 
sponds to physical damage (e.g., a logical error result- 
ing from a software bug that violates a data model rule). 
Consequently, logical inconsistencies can be divided in- 
to two types: (i) logical inconsistencies due to physical 
damage, and (ii) logical inconsistencies dueto violations 
of at least one data model rule (for example, all entity 
property values must be within a rule-specified range, 
an entity must have all of its constituent parts, and item 
must have at least one holding relationship, and so on 
and so forth). 

[0035] In general, "repairing" a logical error is inher- 
ently inferior to "restoring" the data in which the error 
occurs because a backup of that data is likely a good 
copy (or can be used to reconstruct a good copy) of the 
data that was damaged or lost. Therefore, restoration 



techniques are preferred to repair techniques. 
[0036] For several embodiments of the present inven- 
tion, ascertaining whether any entities on a page is log- 
ically corrupted may be accomplished using the ap- 

5 proach illustrated in Fig. 3. For this approach, at step 
302 the LCC checks the database tables for the refer- 
ential integrity of entities existing in the DBFS (the "entity 
externals") and then, at step 304, the LCC checks the 
structural integrity of each entity (the "entity internals," 

10 e.g., constraints and relationships) to ensure no data 
model rule violations. (For certain embodiments, the 
LCC checks every entity in the database.) At step 306, 
the LCC for certain embodiments may also check for 
cycles (e.g., where entity A has a holding relationship to 

15 entity B and entity B has a holding relationship to entity 
A). After the aforementioned checks have been com- 
pleted, the LCC then, at step 308, resolves the logical 
inconsistencies identified. 

[0037] For several embodiments of the present inven- 
20 tion, the LCC utilizes a three-prong approach to resolv- 
ing logical errors as illustrated in Fig. 4. First, at step 
402, the LCC will attempt to perform a page-level res- 
toration (PLR) using the most recent snapshot of the 
page and the transaction log to perfectly restore the 
25 page having the error. However, if a PLR is not possible 
or can not correct the error at step 404, the LCC will then , 
at step 406, attempt an entity-level restoration (ELR) by 
first determining the specific entity that is damaged and 
then restoring that entity from another source (such as 
30 a backup or a sync replica). If both a PLR and an ELR 
are not possible or cannot correct the error at 408, then 
at step 410 the LCC will replace the damaged entity in 
the store with a dummy entity (DE) in order to guarantee 
a consistent view of the file system store as discussed 
35 below. 

[0038] By replacing a damaged entity with a DE, the 
LCC ensures that removal of said damaged entity does 
not corrupt children entities of said damaged entity — t- 
hat is, the LCC prevents cascading corruptions down- 
40 level from the corrupted entity to its children. To accom- 
plish this, the DE essentially replaces the damaged en- 
tity but retains as much information from the damaged 
entity as possible. If the damaged entity is an item, for 
example, the replacing DE will retain as much of the 
45 property data as it can, as well as all of the relationships 
to other items. On the other hand, if the damaged entity 
is a relationship, the replacing DE will continue to con- 
nect the items to which it pertains together. The dam- 
aged entity, meanwhile, is moved to (for items) or logged 
50 in (for relationships) a broken item folder (BIF). When 
the damaged entity is an item, the BIF will have a rela- 
tionship (e.g., a holding relationship) with the damaged 
entity. 

[0039] Figs. 5A and 5B are block diagrams that illus- 
55 trate the replacement methodology regarding items for 
certain embodiments of the present invention. In Fig. 
5A, which illustrates a set of items and relationships, 11 
is a parent item, 12 is a child item of 11 via relationship 
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R12,13 is a child item of 12 via relationship I23, and 14 
and 15 are children items of 13 via relationships R34 and 
R35 respectively. In this example, item 12 is corrupted 
by, for example, a buggy application and, as a result, 
item 12 is now in violation of the data model rules. In Fig. 
5B, the LCC, having identified 12 as a corrupted item, 
first creates DE 12' and establishes a first relationship 
R12 1 between the DE 12' and its parent 11 and a second 
relationship R23' between the DE 12' and its child 13. For 
certain embodiments, the DE is given the same item 
identification number (ItemID) as the corrupted item. 
Corrupted item 12 is then moved to the BIF and with a 
holding relationship Rh between the BIF item and the 
damaged item 12. 

[0040] For certain embodiments, the new relation- 
ships R1 2' and R23' may in fact be the original relation- 
ships R12 and R23 that are updated to associated with 
12' instead of 12. For other embodiments, R12 1 and R23' 
may be entirely new relationships and, for certain such 
embodiments, R12 and R23 may be retained as dan- 
gling relationships with damaged item 12 in the BIF. Re- 
gardless, the DE effectively preserves the parent/child 
structure for the dataset and thereby prevents an error 
to 12 to cascade as errors in 13, 14, and 15 that might 
otherwise be unreachable from 11 . 
[0041] Figs. 6A and 6B are block diagrams that illus- 
trate the replacement methodology regarding relation- 
ships for certain embodiments of the present invention. 
In Fig. 6A, which illustrates a partial set of items and 
relationships, 11 is a parent item, and 12 is a child item 
of 11 via relationship R12. In this example, relationship 
R1 2 is corrupted by, for example, a virus that, as a result, 
causes relationship R12 to have a property value, e.g., 
a security property value, outside of some predeter- 
mined permitted range in the data model. In Fig. 6B, the 
LCC, having identified R12 as a corrupted relationship, 
first creates DE R12' between the 12 and its parent 11 , 
and corrupted relationship R12 is eliminated (and not 
moved to the Bl F since a relationship cannot exist alone 
in the BIF) and item 11, which owned the relationship 
R12, is logged into the BIF (the BIF having a special log 
forthis purpose and shown herein, for example, as a log 
item). 

[0042] In regard to synchronization, and to avoid the 
possibility of a corrupted entity being erroneously syn- 
chronized from partner to partner (thereby spreading the 
corruption), certain embodiments of the present inven- 
tion compartmentalize identified and/or corrupted cor- 
ruptions by marking DEs with a special "non authorita- 
tive" flag (e.g., a singe bit) that effectively notifies any 
sync replica that has a good copy of this entity to over- 
write this entity (at which point the non-authoritative bit 
is cleared). Similarly, if a DE is subsequently modified 
(such as by an end-user), certain embodiments will also 
mark the DE as "non-authoritative and modified" to en- 
sure that a conflict resolution procedure is employed as 
between the modified DE and the good copy of the orig- 
inal item on a replica, and the non-authoritative and 



modified marking will be removed as soon at the conflict 
has been resolved. 

Additional Functionality 

5 

[0043] 

• As described herein, item extensions are consid- 
ered part of the owning item, and thus any extension 

10 corruption is deemed item corruption. However, in 
certain alternative embodiments, extensions may 
be treated as distinct and separate from their own- 
ing items. 

• For certain embodiments of the present invention, 
15 a LCC is run on entities following a restoration op- 
eration performed to correct physical corruption. 

• For certain embodiments, a LCC will attempt to re- 
pair corrupted items first before attempting to cor- 
rect corrupted relationships in order to avoid the de- 

20 tection of "phantom" corruptions that might result if 
an item is not corrected before the relationships that 
depend on it is corrected. 

• For certain embodiments, the BIF is a folder item 
created may be created by the LCC if one does not 

25 already exist in the DBFS. This folder may hold any 
type of item in the DBFS (but not relationships for 
certain embodiments) and the folder may be locat- 
ed off of the root of the default database store (for 
easy access and locating). 
30 • For certain embodiments, any items without back- 
ing files will be put into the BIF, as well as any files 
without corresponding items will also be placed in 
the BIF. 

• For certain embodiments, when a damaged item is 
35 moved to the BIF, the BIF may also store informa- 
tion pertaining to why the damaged item was moved 
to the BIF. 

Conclusion 

40 

[0044] The various system, methods, and techniques 
described herein may be implemented with hardware or 
software or, where appropriate, with a combination of 
both. Thus, the methods and apparatus of the present 

45 invention, or certain aspects or portions thereof, may 
taketheform of program code (i.e., instructions) embod- 
ied in tangible media, such as floppy diskettes, 
CD-ROMs, hard drives, or any other machine-readable 
storage medium, wherein, when the program code is 

50 loaded into and executed by a machine, such as a com- 
puter, the machine becomes an apparatus for practicing 
the invention. In the case of program code execution on 
programmable computers, the computer will generally 
include a processor, a storage medium readable by the 

55 processor (including volatile and non-volatile memory 
and/or storage elements), at least one input device, and 
at least one output device. One or more programs are 
preferably implemented in a high level procedural orob- 
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ject oriented programming language to communicate 
with a computer system. However, the program(s) can 
be implemented in assembly or machine language, if 
desired. In any case, the language may be a compiled 
or interpreted language, and combined with hardware 
implementations. 

[0045] The methods and apparatus of the present in- 
vention may also be embodied in the form of program 
code that is transmitted over some transmission medi- 
um, such as over electrical wiring or cabling, through 
fiber optics, or via any other form of transmission, 
wherein, when the program code is received and loaded 
into and executed by a machine, such as an EPROM, a 
gate array, a programmable logic device (PLD), a client 
computer, a video recorder or the like, the machine be- 
comes an apparatus for practicing the invention. When 
implemented on a general-purpose processor, the pro- 
gram code combines with the processor to provide a 
unique apparatus that operates to perform the indexing 
functionality of the present invention. 
[0046] While the present invention has been de- 
scribed in connection with the preferred embodiments 
of the various figures, it is to be understood that other 
similar embodiments may be used or modifications and 
additions may be made to the described embodiment 
for performing the same function of the present inven- 
tion without deviating there from. For example, while ex- 
emplary embodiments of the invention are described in 
the context of digital devices emulating the functionality 
of personal computers, one skilled in the art will recog- 
nize that the present invention is not limited to such dig- 
ital devices, as described in the present application may 
apply to any number of existing or emerging computing 
devices or environments, such as a gaming console, 
handheld computer, portable computer, etc. whether 
wired or wireless, and may be applied to any number of 
such computing devices connected via a communica- 
tions network, and interacting across the network. Fur- 
thermore, it should be emphasized that a variety of com- 
puter platforms, including handheld device operating 
systems and other application specific hardware/soft- 
ware interface systems, are herein contemplated, espe- 
cially as the number of wireless networked devices con- 
tinues to proliferate. Therefore, the present invention 
should not be limited to any single embodiment, but rath- 
er construed in breadth and scope in accordance with 
the appended claims. 



Claims 

1 . A logical consistency checker (LCC) for a database 
file system (DBFS), said LCC comprising at least 
one system for: 

checking referential integrity of a set of tables 
in a database corresponding to said DBFS for 
at least one logical inconsistency; and 



checking structural integrity of a set of entities 
represented in said DBFS for at least one log- 
ical inconsistency. 

5 2. The system of claim 1 further comprising at least 
one subsystem for checking for at least one cycle 
that would constitute a logical inconsistency. 

3. The system of claim 1 further comprising at least 
10 one subsystem for resolving at least one logical in- 
consistency. 

4. The system of claim 3 wherein said at least one sub- 
system for resolving at least one logical inconsist- 

15 ency attempts at least one solution from among the 
following groups of solutions: a page-level restora- 
tion, a entity-level restoration, and replacing a dam- 
aged entity with a dummy entity. 

20 5. The system of claim 4 wherein said at least one sub- 
system for resolving at least one logical inconsist- 
ency first attempts a page-level restoration and, if 
not successful, then attempts an entity-level resto- 
ration and, if not successful, then replaces at least 
25 one damaged entity corresponding to said at least 
one logical inconsistency with a dummy entity. 

6. The system of claim 4 wherein, if said entity is an 
item, said solution of replacing a damaged entity 

30 with a dummy entity comprises: 

creating said dummy entity and redirecting at 
least one relationship belonging to said dam- 
aged entity with said dummy entity; and 
35 moving said damaged entity to a broken item 

folder. 

7. The system of claim 4 wherein, if said entity is a 
relationship, said solution of replacing a damaged 

40 entity with a dummy entity comprises: 

creating said dummy entity as a relationship 
corresponding to the relationship of the dam- 
aged item; and 
45 logging said damaged entity in a broken item 

folder. 

8. An automated data reliability system (DRS) for a da- 
tabase file system (DBFS), said DRS comprising: 

50 

a set of policies for performing database admin- 
istration (DBA) tasks; 

a subsystem for resolving a set of physical data 
corruptions at a page level; and 
55 a subsystem for resolving a set of logical data 

corruptions at an entity level. 

9. The system of claim 8 wherein said DBFS is a com- 
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ponent of an item-based hardware/software inter- 
face system. 

The system of claim 8 further comprising an inter- 
face for adding, deleting, and modifying at least one 5 
functionality from among the following group of 
functionalities: error and event notifications, poli- 
cies, and error/event handling algorithms. 



if said attempt at an entity-level restoration is 
not successful, then replacing at least one dam- 
aged entity corresponding to said at least one 
logical inconsistency with a dummy entity. 

19. The method of claim 1 7 wherein, if said entity is an 
item, said solution of replacing a damaged entity 
with a dummy entity further comprises: 



11. The system of claim 8 wherein said DRS operates 
as a background thread. 

12. The system of claim 8 wherein said subsystem for 
resolving a set of logical data corruptions at an en- 
tity level is executed after the execution of said sub- 
system for resolving a set of physical data corrup- 
tions at a page level. 

13. The system of claim 8 wherein said subsystem for 
resolving a set of logical data corruptions at an en- 
tity level attempts to repair a corrupted item before 
attempting to correct a corrupted relationship cor- 
responding to said corrupted item. 

14. A method for a logical consistency checker (LCC) 
for a database file system (DBFS) to address logical 
inconsistencies on an entity-level, said method 
comprising: 

checking referential integrity of a set of tables 
in a database corresponding to said DBFS for 
at least one logical inconsistency; and 
checking structural integrity of a set of entities 
represented in said DBFS for at least one log- 
ical inconsistency. 

15. The method of claim 14furthercomprising checking 
for at least one cycle that would constitute a logical 
inconsistency. 

16. The method of claim 14further comprising resolving 
at least one logical inconsistency. 

17. The method of claim 1 6 wherein said element of re- 
solving at least one logical inconsistency comprises 
the attempt of at least one solution from among the 
following groups of solutions: a page-level restora- 
tion, a entity-level restoration, and replacing a dam- 
aged entity with a dummy entity. 

18. The method of claim 1 7 wherein said element of re- 
solving at least one logical inconsistency compris- 
es: 

attempting at a page-level restoration; 
if said attempt at a page-level restoration is not 
successful, then attempting an entity-level res- 
toration; and 



10 creating said dummy entity and redirecting at 

least one relationship belonging to said dam- 
aged entity with said dummy entity; and 
moving said damaged entity to a broken item 
folder. 

15 

20. The method of claim 17 wherein, if said entity is a 
relationship, said solution of replacing a damaged 
entity with a dummy entity comprises: 

20 creating said dummy entity as a relationship 

corresponding to the relationship of the dam- 
aged item; and 

logging said damaged entity in a broken item 
folder. 

25 

21. A method for an automated data reliability system 
(DRS) for a database file system (DBFS) to address 
logical inconsistencies on an entity-level, said 
method comprising: 

30 

utilizing a set of policies for performing data- 
base administration (DBA) tasks; 
resolving a set of physical data corruptions at 
a page level; and 
35 resolving a set of logical data corruptions at an 

entity level. 

22. The method of claim 21 wherein said DBFS is a 
component of an item-based hardware/software in- 

40 terface system. 

23. The method of claim 21 further comprising an inter- 
face for adding, deleting, and modifying at least one 
functionality from among the following group of 

45 functionalities: error and event notifications, poli- 
cies, and error/event handling algorithms. 

24. The method of claim 21 wherein said DRS operates 
as a background thread. 

50 

25. The method of claim 21 wherein said element of re- 
solving a set of logical data corruptions at an entity 
level is executed after the execution of said subsys- 
tem for resolving a set of physical data corruptions 

55 at a page level. 

26. The method of claim 21 wherein said element of re- 
solving a set of logical data corruptions at an entity 
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level further comprises attempting to repair a cor- 
rupted item before attempting to correct a corrupted 
relationship corresponding to said corrupted item. 

27. A computer-readable medium comprising compu- 
ter-readable instructions for a logical consistency 
checker (LCC) for a database file system (DBFS) 
to address logical inconsistencies on an entity-lev- 
el, said computer-readable instructions comprising 
instructions for: 

checking referential integrity of a set of tables 
in a database corresponding to said DBFS for 
at least one logical inconsistency; and 
checking structural integrity of a set of entities 
represented in said DBFS for at least one log- 
ical inconsistency. 

28. The computer-readable instructions of claim 27 fur- 
ther comprising instructions for checking for at least 
one cycle that would constitute a logical inconsist- 
ency. 

29. The computer-readable instructions of claim 27 fur- 
ther comprising instructions for resolving at least 
one logical inconsistency. 

30. The computer-readable instructions of claim 29 fur- 
ther comprising instructions whereby said element 
of resolving at least one logical inconsistency com- 
prises the attempt of at least one solution from 
among the following groups of solutions: a page- 
level restoration, a entity-level restoration, and re- 
placing a damaged entity with a dummy entity. 

31 . The computer-readable instructions of claim 30 fur- 
ther comprising instructions whereby said element 
of resolving at least one logical inconsistency com- 
prises: 

attempting at a page-level restoration; 
if said attempt at a page-level restoration is not 
successful, then attempting an entity-level res- 
toration; and 

if said attempt at an entity-level restoration is 
not successful, then replacing at least one dam- 
aged entity corresponding to said at least one 
logical inconsistency with a dummy entity. 

32. The computer-readable instructions of claim 30 fur- 
ther comprising instructions whereby, if said entity 
is an item, said solution of replacing a damaged en- 
tity with a dummy entity further comprises: 

creating said dummy entity and redirecting at 
least one relationship belonging to said dam- 
aged entity with said dummy entity; and 
moving said damaged entity to a broken item 



folder. 

33. The computer-readable instructions of claim 30 fur- 
ther comprising instructions whereby, if said entity 

5 is a relationship, said solution of replacing a dam- 

aged entity with a dummy entity comprises: 

creating said dummy entity as a relationship 
corresponding to the relationship of the dam- 
10 aged item; and 

logging said damaged entity in a broken item 
folder. 

34. A computer-readable medium comprising compu- 
15 ter-readable instructions for an automated data re- 
liability system (DRS) for a database file system 
(DBFS) to address logical inconsistencies on an en- 
tity-level, said computer-readable instructions com- 
prising instructions for: 

20 

utilizing a set of policies for performing data- 
base administration (DBA) tasks; 
resolving a set of physical data corruptions at 
a page level; and 
25 resolving a set of logical data corruptions at an 

entity level. 

35. The computer-readable instructions of claim 34 fur- 
ther comprising instructions whereby said DBFS is 

30 a component of an item-based hardware/software 
interface system. 

36. The computer-readable instructions of claim 34 fur- 
ther comprising instructions for an interface for add- 

35 ing, deleting, and modifying at least one functional- 
ity from among the following group of functionalities: 
error and event notifications, policies, and error/ 
event handling algorithms. 

40 37. The computer-readable instructions of claim 34 fur- 
ther comprising instructions whereby said DRS op- 
erates as a background thread. 



38. The computer-readable instructions of claim 34 f ur- 
45 ther comprising instructions whereby said element 

of resolving a set of logical data corruptions at an 
entity level is executed after the execution of said 
subsystem for resolving a set of physical data cor- 
ruptions at a page level. 

50 

39. The computer-readable instructions of claim 34 fur- 
ther comprising instructions whereby said element 
of resolving a set of logical data corruptions at an 
entity level further comprises attempting to repair a 

55 corrupted item before attempting to correct a cor- 
rupted relationship corresponding to said corrupted 
item. 
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40. A hardware control device for an automated data 
reliability system (DRS) for a database file system 
(DBFS) to address logical inconsistencies on an en- 
tity-level, said device comprising means for: 

utilizing a set of policies for performing data- 
base administration (DBA) tasks; 
resolving a set of physical data corruptions at 
a page level; and 

resolving a set of logical data corruptions at an 
entity level by: 

checking referential integrity of a set of ta- 
bles in a database corresponding to said 
DBFS for at least one logical inconsistency, 
and 

checking structural integrity of a set of en- 
tities represented in said DBFS for at least 
one logical inconsistency. 
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